Distributional reinforcement learning with unconstrained monotonic neural networks

The distributional reinforcement learning (RL) approach advocates for representing the complete probability distribution of random return instead only modelling its expectation. A RL algorithm may be characterised by two main components, namely representation together with parameterisation and metric defining loss. present research work considers unconstrained monotonic neural network (UMNN) architecture, a universal approximator continuous functions which is particularly well suited different representations distribution. This property enables efficient decoupling effect function class from that metric. paper firstly introduces methodology (PDF, CDF QF). Secondly, novel named deep Q-network (UMDQN) presented. To authors' knowledge, it first method supporting three, valid Lastly, in light this new algorithm, an empirical comparison performed between three quasi-metrics, Kullback-Leibler divergence, Cramer distance, Wasserstein distance. results highlight strengths weaknesses associated each important limitation